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(54) Image coding apparatus and image decoding apparatus 

(57) An image coding apparatus which includes a 
frame memory selecting unit (35) for selecting, in 
response to a selection signal, an image to be continu- 
ously stored in a plurality of frame memories (9, 10) as 
a background image and storing the background image 
into the plurality of frame memories (9, 1 0), and a back- 
ground motion compensating unit (14, 39) for perform- 
ing motion compensating prediction corresponding to 
an input image based on the background image to gen- 
erate a predicted image based on the motion compen- 
sating prediction, and an image decoding apparatus 
corresponding to the image coding apparatus. 
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1 EPOJ 

Description 

BACKGROUND OF THE INVENTION 

Field of the Invention 

This invention relates to an image coding apparatus 
and an image decoding apparatus for use with a system 
which performs high efficiency coding or decoding of 
moving pictures to perform efficient transmission or 
storage of images, and more particularly to an image 
coding apparatus and an image decoding apparatus 
which can be applied to processing of, for example, a 
digital broadcasting system which is performed using a 
satellite or a ground wave or cable communication net- 
work, a digital video disk, a mobile video phone, a PHS 
video phone or a data base for images. 

Description of the Prior Art 

As a representative one of conventional high effi- 
ciency coding systems, the MPEG2 is known which is 
an international standard system recommended by the 
ISO/IEC/JTC1/SC29/WG1 1. For example, "Image Infor- 
mation Engineering and Broadcasting Techniques", 
Journal of the Television Engineering Society of Japan, 
April, 1995 explains the MPEG as a theme of special 
editing. A coding system of the MPEG2 is disclosed in 
"3-2 Video Compression" of the same document, pp. 
29-60. 

The coding system of the MPEG2 is described 
below. 

FIG. 31 is a block diagram showing a basic con- 
struction of an ordinary encoder of the MPEG2, and 
FIG. 32 is a block diagram showing a basic construction 
of an MPEG2 decoder. Referring to FIGS. 31 and 32, 
reference numeral 1 denotes a frame rearranging unit, 2 
a subtracting unit, reference characters 3a and 3b 
denote each an inter(interframe)/intra(intraframe) 
switching selector, reference numeral 4 denotes a con- 
verting unit, 5 a quantizing unit, 6 a reverse quantizing 
unit, 7 a reverse converting unit, 8 an adder, 9 a first 
frame memory, 10 a second frame memory, 11 a for- 
ward direction motion compensating unit, 12 a bidirec- 
tion motion compensating unit, 13 a backward direction 
motion compensating unit, 15 a motion estimating unit, 
16 a coding control unit, 17 a variable length coding 
unit, and 18 a buffer. 

Further, reference numeral 100 denotes input 
image data in the form of digital data, 101 re-arranged 
input image data, 102 a predictive error image, 103 an 
original input image or predictive error image, 104 a 
conversion coefficient, 105 a quantization coefficient, 
106 a reverse quantized conversion coefficient, 107 
reversed converted image data, 108 a locally decoded 
image, 109 a reference image from the first frame mem- 
ory, 110 a reference image from the second frame 
memory, 111 a forward direction motion predicted 
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image, 112 a bidirection motion predicted image, 113 a 
backward direction motion predicted image, 115 a 
determined predicted image, 117 a control signal to the 
selector, 118 a control signal to the converting unit 4, 
s 119 an adaptive quantization value, 120 a variable 
length coder, 121 a bit stream, 123 a motion vector, 124 
a reference image, and 125 an intra/inter switching sig- 
nal. 

Operation of the conventional image encoder is 
10 described below with reference to FIG. 31 . 

First, an input image signal 1 00 in the form of a dig- 
ital signal is inputted to the frame re-arranging unit 1 , by 
which picture frames to be coded are re-arranged. 

FIG. 33 illustrates such re-arrangement. Referring 
is to FIG. 33, reference character I denotes an intra (intra- 
frame) coded picture, P an interframe coded picture, 
and B a bidirectional predictive coded picture. It is to be 
noted that reference numerals 1 to 10 represent an 
order in time in which they are displayed. 
20 The first frame is first coded as an I picture, and 
then the fourth frame is coded as a P picture, where- 
upon the already coded I picture is used as a reference 
frame for prediction. 

Then, the second frame is coded as a B picture. 
25 Thereupon, the I picture of the first frame and the P pic- 
ture of the fourth frame coded already are used as ref- 
erence frames for the prediction. In FIG. 33, each arrow 
mark represents a direction in which prediction is per- 
formed. 

30 Thereafter, coding is performed in the construction 
of I, B, B, P, B, B, P, ... by similar processing. Accord- 
ingly, the action of the frame re-arranging unit 1 is to re- 
arrange the input image signal 1 00, in which the picture 
frames are arranged in order of time, so that they 

35 appear in order of coding in order to allow the process- 
ing described above. 

Subsequently, since predictive coding is not per- 
formed for the I picture mentioned above, when the re- 
arranged image 101 is inputted as it is to the selector 

40 3a, from which it is transmitted as a selector output 103 
to the converting unit 4. On the other hand, for predictive 
coding for the P picture or the B picture mentioned 
above, the re-arranged image 101 is subtracted from a 
predicted image 115 by the subtracting unit 2, and a 

45 predictive error image 1 02 is transmitted as the selector 
output 103 to the converting unit 4. 

Then, the selector output 1 03 is inputted to the con- 
verting unit 4, and a conversion coefficient 104 is out- 
putted from the converting unit 4. The conversion 

so coefficient 104 passes the quantizing unit 5, and a 
quantization coefficient 105 is obtained from the quan- 
tizing unit 5. The quantization coefficient 105 is coded 
into a variable length code by the variable length coding 
unit 17, and a variable length coded word 120 is output- 

55 ted from the variable length coding unit 17. 

The quantization coefficient 105 is, on the other 
hand, inputted to the reverse quantizing unit 6, and a 
quantization coefficient 106 is outputted from the 
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reverse quantizing unit 6. 

Further, the quantization coefficient 106 is reverse 
converted back to an image level by the reverse con- 
verting unit 7, and image data 107 is outputted from the 
reverse converting unit 7. The image data 1 07 is, where 5 
it is data of the I picture, added to a predicted image 1 1 6 
selected by the adding unit 8, and a locally decoded 
image 108 is outputted from the adding unit 8. 

It is to be noted that the locally decoded image 108 
is written as it is into the first frame memory 9 when it is 10 
an I picture, but, when it is a P picture, it is written into 
the second frame memory 10. 

On the other hand, when the locally decoded image 
1 08 is a B picture, it is written into none of the first frame 
memory 9 and the second frame memory 10. 15 

Thereafter, when the locally decoded image 108 is 
a P picture, since it is used only for forward direction 
prediction, a reference image 124 in the first frame 
memory 9 is read out, and motion prediction is per- 
formed for each macroblock (basic unit for processing of so 
16 pixels x 16 lines) by the motion estimating unit 15. 
The motion estimating unit 15 thus selects one of the 
macroblocks which has a value nearest to that of the 
current macroblock as a predicted image, and simulta- 
neously outputs a motion vector 1 23 therefrom. ss 

The motion vector 123 is inputted to the motion 
compensating units 1 1 , 12 and 13 surrounded by a dot- 
ted line in FIG, 31, and motion predictive pictures are 
outputted from the motion compensating units 11, 12 
and 13. 30 

In this instance, the forward direction motion com- 
pensating unit 1 1 produces a forward direction motion 
predicted image 1 1 1 using a reference image 109 from 
the first frame memory 9 and outputs a thus determined 
predicted image 115. 35 

Further, as described hereinabove, the locally 
decoded images 108 of all macroblocks in a P picture 
are written into the second frame memory. However, 
even with the P picture mentioned above, when the 
macroblocks thereof are intraframe (intra) coded, the 40 
frame re-arranged image 101 is outputted directly as 
the selector output. 

Meanwhile, for a B picture, the procedure of coding 
processing is similar to that for a P picture described 
above, but different from the processing for a P picture, 4s 
two reference frames are used for prediction. 

The motion estimating unit 15 performs forward 
direction prediction using the reference image 109 from 
the first frame memory 9, backward direction prediction 
using a reference image 110 from the second frame so 
memory 10, and bidirection prediction using both of the 
reference images 1 09 and 1 1 0 to select one of the pre- 
diction modes with which a value nearest to that of the 
current macroblock is obtained, and outputs a motion 
vector 123 then. 55 

In accordance with the thus determined prediction 
mode, in the motion compensating unit, one of the 
motion compensating units 11, 12 and 13 which corre- 



sponds to the determined prediction mode produces 
and outputs a predicted picture. 

For example, when bidirection motion prediction is 
selected, the bidirection motion compensating unit 12 
produces and outputs a predicted image 115 deter- 
mined using a bidirection predicted image 112. 

After coding of the B pictures of the second and 
third frames shown in FIG. 33 is completed, image data 
written in the second frame memory is transferred to the 
first frame memory. Thereafter, the P picture of the sev- 
enth frame is coded, and a decoded picture is written 
into the second frame memory. 

Thereafter, B pictures (fifth and sixth frames) are 
coded by similar processing to that described above. 

When the macroblocks are intraframe (intra) coded, 
the image 101 after frame re-arrangement is directly 
outputted as the selector output similarly as in the case 
of a P picture. 

FIG. 32 is a block diagram of a conventional 
decoder. Referring to FIG. 32, reference character 22 
denotes a variable length decoding unit, 107(a) an intra 
(intraframe) coded picture, and 107(b) a prediction error 
picture. 

Subsequently, operation of the conventional 
decoder will be described. 

A bit stream 121 is stored for a certain period of 
time into the receiving buffer 18, and a variable length 
coded word 120 is variable length decoded by the vari- 
able length decoding unit 22 and outputted as a quanti- 
zation coefficient 105. 

The processing procedure after this is quite similar 
to the local decoding processing of the encoder 
described hereinabove. 

When the macroblock is intra decoded, a reverse 
converted image 107 makes an image 107(a) without 
passing the adding unit 8, but when the macroblock is 
inter (interframe) decoded, the reverse converted image 
data 107 makes an image 107(b). The image 107(b) is 
added to a predicted image 1 15 by the adding unit 8, 
and a decoded image 108 is outputted from the adding 
unit 8. The decoded image 108 is processed by the dis- 
played frame re-arranging unit 38 such that such 
decoded images are re-arranged so that they appear in 
order of time, and finally, an output image 1 37 is output- 
ted the displayed frame re-arranging unit 38. 

The example of a conventional image coder and 
image decoder described above is a representative 
apparatus of a type which performs forward direction, 
bidirection and backward direction prediction coding in 
combination. 

In the example, for coding of a P picture, only for- 
ward direction prediction is performed using the first 
frame memory to perform predictive coding. On the 
other hand, for coding of a B picture, one of the modes 
of forward direction prediction, backward direction pre- 
diction and bidirection prediction with which a minimum 
predictive error is provided is selected using the first 
and second frame memories. 
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Accordingly, as coding processing proceeds, 
decoded pictures written in the frame memories are 
erased. Consequently, for example, even if one of coded 
pictures processed in the past is similar to a picture of 
the currently coded frame, since the past decoded pic- 
tures have already been erased from the frame memo- 
ries, the similar coded picture cannot be used for 
reference, resulting in a subject to be solved in that the 
efficiency in image processing is low as much. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide, in 
order to solve the subject of the conventional image 
encoder and image decoder described above, an image 
coding apparatus and an image decoding apparatus 
wherein a decoded image obtained in the past can be 
utilized efficiently as a reference picture and the overall 
prediction efficiency is improved to achieve moving pic- 
ture coding and decoding of a high efficiency. 

According to an aspect of the present invention, 
there is provided an image coding apparatus, compris- 
ing storage means for storing a plurality of decoded 
images, motion compensating prediction means for per- 
forming motion compensating prediction corresponding 
to an input image based on the plurality of decoded 
images stored in the storage means to produce a 
motion vector and for generating a predicted image 
based on the motion compensating prediction, predic- 
tion error calculation means for calculating a difference 
between the predicted image generated by the motion 
compensating prediction means and the input image to 
calculate a prediction error image, decoding means for 
generating the decoded images from the prediction 
error image calculated by the prediction error calcula- 
tion means and the predicted image, background image 
storage control means for selecting one of the decoded 
images which is to be continuously stored in the storage 
means as a background image and storing the back- 
ground image into the storage means, and background 
motion compensation means for performing motion 
compensating prediction corresponding to the input 
image based on the background image to generate a 
motion vector and generating a predicted image based 
on the motion compensating prediction. 

According to another aspect of the present inven- 
tion, there is provided an image decoding apparatus, 
comprising storage means for storing a plurality of 
decoded images, motion compensation means for per- 
forming motion compensating prediction based on the 
decoded images stored in the storage means to gener- 
ate a motion compensated image, decoding means for 
generating the coded images from the motion compen- 
sated image from the motion compensation means and 
a prediction error image, background image storage 
control means for selecting one of the decoded images 
which is to be continuously stored in the storage means 
as a background image and storing the background 



image into the storage means, and background pre- 
dicted image generation means for generating a back- 
ground predicted image based on the background 
image. 

5 The image coding apparatus and the image decod- 
ing apparatus of the present invention may be con- 
structed such that the storage means includes a frame 
memory for storing a decoded image, and another 
frame memory for storing the background image. 

10 The image coding apparatus and the image decod- 
ing apparatus of the present invention may otherwise be 
constructed such that re-writing of image contents into 
the storage means by the background image storage 
control means is performed in units of a picture after a 

w predetermined interval of time or in response to a con- 
trol signal from the outside. 

The image coding apparatus and the image decod- 
ing apparatus of the present invention may otherwise be 
constructed such that re-writing of image contents into 

20 the storage means by the background image storage 
control means is performed in units of a macroblock 
after a predetermined interval of time or in response to 
a control signal from the outside. 

The image coding apparatus of the present inven- 

ss tion may otherwise be constructed such that the back- 
ground motion compensation means has a variable 
searching range for a motion vector from the back- 
ground images. 

The image coding apparatus of the present inven- 

30 tion may otherwise be constructed such that it further 
comprises differential vector generation means for hold- 
ing a motion vector obtained from the motion compen- 
sating prediction means or the background motion 
compensation means and calculating a difference vec- 

35 tor between the generated motion vector and the motion 
vector in the past, and the difference vector is variable 
length coded. 

The image decoding apparatus of the present 
invention may otherwise be constructed such that it fur- 

40 ther a motion vector adding unit for holding a motion 
vector decoded in the past and adding the motion vector 
decoded in the past to a difference vector to regenerate 
a motion vector. 

According to a further aspect of the present inven- 
ts tion, there is provided an image decoding apparatus 
which outputs a coded bit stream of moving pictures, 
comprising a plurality of frame memory groups for stor- 
ing, individually for a plurality of objects which compose 
a screen, decoded images of the objects in the past, a 

so frame memory selecting unit for selecting, in response 
to a control signal, into a frame memory of which one of 
the plurality of frame memory groups a decoded image 
is to be written, a motion compensation predicting unit 
for selecting one of forward direction prediction, back- 

55 ward direction prediction, bidirection prediction and 
background prediction in units of an object using refer- 
ence images read out from frame memories of the plu- 
rality of frame memory groups provided for the 
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individual objects to perform motion compensating pre- 
diction, a subtracter for calculating a difference between 
the predicted image and a current image to calculate a 
prediction error image, an adding unit for adding the 
predicted image from the reference images and the pre- 5 
diction error image of the current image, and a variable 
length coding unit for variable length coding information. 

According to a still further aspect of the present 
invention, there is provided an image decoding appara- 
tus which decodes a coded bit stream of moving pic- 10 
tures, comprising a plurality of frame memory groups for 
storing, individually for a plurality of objects which con- 
struct a screen, decoded images of the objects, a frame 
memory selecting unit for selecting, in response to a 
control signal, into a frame memory of which one of the is 
plurality of frame memory groups the coded images are 
to be written for the individual objects, a variable length 
decoding unit for variable length decoding the coded bit 
stream, and a motion compensating unit for selecting 
one of forward direction prediction, backward direction 20 
prediction, bidirection prediction and background pre- 
diction in units of an object using reference images read 
out from frame memories of the plurality of frame mem- 
ory groups to generate a motion compensated image. 

The image coding apparatus or the image decoding 25 
apparatus of the present invention may be constructed 
such that the plurality of frame memory groups include 
three frame memory groups. 

The image coding apparatus of the present inven- 
tion may otherwise be constructed such that re-writing 30 
of image contents of a region in which an object which 
is a subject of coding is included in the plurality of frame 
memory groups in which coded images of the object in 
the past are stored is performed after a certain interval 
of time or in response to a control signal from the out- 35 
side. 

The image decoding apparatus of the present 
invention may otherwise be constructed such that re- 
writing of image contents of a region in which an object 
which is a subject of decoding is included in the plurality 40 
of frame memory groups in which coded images of the 
object in the past are stored is performed after a certain 
interval of time or in response to a control signal from 
the outside. 

The image coding apparatus of the present inven- 45 
tion may otherwise be constructed such that searching 
ranges for a motion vector from reference images from 
the plurality of frame memory groups for the individual 
objects are variable for the individual objects. 

The image coding apparatus of the present inven- so 
tion may otherwise be constructed such that it further 
comprises differential vector generation means for hold- 
ing a motion vector in the past obtained by referring to 
images from the plurality of frame memory groups for 
the individual objects and calculating difference vectors 55 
separately for the individual objects, and the difference 
vectors are variable length coded. 

The image decoding apparatus of the present 



invention may otherwise be constructed such that it fur- 
ther comprises a motion vector adding unit for holding 
decoded motion vectors in the past obtained by refer- 
ring to images in the plurality of frame memory groups 
for the individual objects for a certain period of time and 
adding the motion vectors decoded in the past to the 
decoded difference vectors to regenerate motion vec- 
tors for the individual objects. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a coding apparatus for 
moving pictures according to an embodiment 1 of 
the present invention; 

FIG. 2 is a block diagram showing an internal con- 
struction of a motion estimating unit of the coding 
apparatus of the embodiment 1 of the present 
invention; 

FIG. 3 is a block diagram showing an internal con- 
struction of a motion compensation predicting unit 
of the coding apparatus of the embodiment 1 of the 
present invention; 

FIG. 4 is a block diagram showing another con- 
struction of the coding apparatus for moving pic- 
tures according to the embodiment 1 of the present 
invention; 

FIGS. 5A to 5C are diagrammatic views illustrating 
an example of the relationship between patterns of 
pictures and prediction modes in the embodiment 1 
of the present invention; 

FIG. 6 is a block diagram showing a further con- 
struction of the coding apparatus for moving pic- 
tures according to the embodiment 1 of the present 
invention; 

FIG. 7 is a block diagram of a decoding apparatus 
for moving pictures according to an embodiment 2 
of the present invention; 

FIG. 8 is a block diagram of a motion compensating 
unit of the decoding apparatus of the embodiment 2 
of the present invention; 

FIG. 9 is a block diagram of a coding apparatus for 
moving pictures according to an embodiment 3 of 
the present invention; 

FIG. 10 is a block diagram of a motion estimating 
unit of the coding apparatus in the embodiment 3 of 
the present invention; 

FIG. 1 1 is a block diagram of a motion compensat- 
ing unit of the coding apparatus in the embodiment 
3 of the present invention; 
FIGS. 12A, 12B and 12C are diagrammatic views 
illustrating an example of the relationship between 
picture patterns and prediction modes in the 
embodiment 3 of the present invention; 
FIG. 13 is a block diagram of a decoding apparatus 
for moving pictures according to an embodiment 4 
of the present invention; 

FIG. 14 is a block diagram of a motion compensat- 
ing unit of the decoding apparatus of the embodi- 
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ment 4 of the present invention; 
FIG. 15 is a diagrammatic view illustrating re-writing 
of a picture in a frame memory in units of a macrob- 
lock in a coding apparatus according to an embodi- 
ment 5 of the present invention ; s 
FIG. 16 is a block diagram of a coding apparatus 
according to an embodiment 8 of the present inven- 
tion; 

FIGS. 17A and 17B are diagrammatic views illus- 
trating a coding method of a motion vector in the 10 
embodiment 8 of the present invention; 
FIG. 18 is a block diagram showing another con- 
struction of the coding apparatus according to the 
embodiment 8 of the present invention; 
FIG. 1 9 is a block diagram of a decoding apparatus is 
according to an embodiment 9 of the present inven- 
tion; 

FIG. 20 is a block diagram showing another con- 
struction of the decoding apparatus according to 
the embodiment 9 of the present invention; 20 
FIG. 21 is a diagrammatic view illustrating the rela- 
tionship between pictures and objects; 
FIG. 22 is a block diagram of a coding apparatus 
according embodiments 10 and 15 of the present 
invention; ss 
FIG. 23 is a block diagram of a decoding apparatus 
according to an embodiment 11 of the present 
invention; 

FIG. 24 is a block diagram of a coding apparatus 
according to embodiments 12 and 1 5 of the present 30 
invention; 

FIG. 25 is a block diagram of a decoding apparatus 
according to an embodiment 13 of the present 
invention; 

FIG. 26 is a diagrammatic view illustrating re-writing 35 
of an image in an object region performed in a cod- 
ing apparatus according to an embodiment 14 of 
the present invention; 

FIG. 27 is a diagrammatic view of a coding appara- 
tus according to an embodiment 16 of the present 40 
invention; 

FIG. 28 is a diagrammatic view of a decoding appa- 
ratus according to an embodiment 1 7 of the present 
invention; 

FIG. 29 is a diagrammatic view of a coding appara- 4S 
tus according to an embodiment 18 of the present 
invention; 

FIG. 30 is a diagrammatic view of a decoding appa- 
ratus according to an embodiment 1 9 of the present 
invention; so 
FIG. 31 is a block diagram of a conventional 
encoder; 

FIG. 32 is a block diagram of a conventional 
decoder; and 

FIG. 33 is a diagrammatic view showing an exam- 55 
pie of an array of pictures. 



DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

In the following, image coding apparatus and image 
decoding apparatus of preferred embodiments of the 
present invention will be described with reference to the 
accompanying drawings. 

Embodiment 1. 

FIG. 1 is a block diagram of a coding apparatus for 
moving pictures according to an embodiment 1 of the 
present invention. Referring to FIG. 1, reference 
numeral 21 denotes a motion compensation predicting 
unit as motion compensation predicting means, 35 a 
frame memory selecting unit as background image stor- 
ing control means, and 45 a multiplexing unit. Further, 
reference numeral 126 denotes a determined motion 
prediction mode, reference numerals 134 and 135 
denote each a selected decoded image, and reference 
numeral 139 denotes a multiplexed bit stream. Since 
the other components are similar to those used in the 
description of the prior art shown in FIGS. 31 to 33, they 
are denoted by same reference numerals and repetitive 
description of them is omitted here. 

Subsequently, operation will be described. 

Basic coding operation is equivalent to motion com- 
pensating prediction + conversion coding described 
hereinabove in connection with the conventional exam- 
ple. Accordingly, only differences will be described here. 

A locally decoded image 108 is inputted to the 
frame memory selecting unit 35, by which it is selected 
into which one of the first frame memory 9 and the sec- 
ond frame memory 10 it is to be written. Meanwhile, the 
motion estimating unit 15 reads out reference images 
109 and 110 from the frame memories 9 and 10 and 
outputs a determined motion prediction mode 1 26 and a 
motion vector 123 with which the prediction error of the 
locally decoded image 108 from the re-arranged input 
image data 101 is minimized. 

The motion compensation predicting unit 21 reads 
out the reference images 109 and 110 and outputs a 
motion predicted image 115 based on the determined 
motion prediction mode 126 and the motion vector 123. 

The bit stream 121 is multiplexed together with the 
prediction mode 126 by the multiplexing unit 45 and for- 
warded from the multiplexing unit 45. 

The foregoing is the basic operation of the image 
coding apparatus of the embodiment 1 . In the following, 
details of the individual units will be described. 

FIG. 2 shows an internal construction of the motion 
estimating unit 15. Referring to FIG. 2, reference 
numeral 27 denotes a forward direction predicted image 
generating unit, 28 a bidirection predicted image gener- 
ating unit 29 a backward direction predicted image gen- 
erating unit, 30 a background predicted image 
generating unit, 31 a prediction mode determining unit, 
127 a forward direction predicted image, 128 a bidirec- 
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tion predicted image, 129 a backward direction pre- 
dicted image, and 130 a background predicted image. 

Subsequently, operation of the motion estimating 
unit 15 will be described. 

Each of the predicted image generating units 27, 
28, 29 and 30 generates a predicted image in accord- 
ance with a predetermined prediction mode. 

For example, the forward direction predicted image 
generating unit 27 reads out reference images 109 from 
the first frame memory 9 and searches the reference 
images 109 for an image which has a value nearest to 
the value of the input image data 101. 

To this end, for example, a block matching method 
which is employed also in the conventional example 
described in connection with the prior art may be used 
as it is. In particular, matching is performed for all pixels 
in macroblocks described above, and an image wherein 
the sum total of error values exhibits a minimum value is 
searched for. As a result, the forward direction predicted 
image generating unit 27 outputs a forward direction 
predicted image 127. 

The backward direction predicted image generating 
unit 29 performs searching of reference images 110 
from the second frame memory 10 and then performs 
block matching similarly. Then, the backward direction 
predicted image generating unit 29 outputs a backward 
direction predicted image 129. 

The bidirection predicted image generating unit 28 
outputs a bidirection predicted image 128 using the two 
frame memories 9 and 10. The bidirection predicted 
image generating unit 28 generates a forward direction 
predicted image and a backward direction predicted 
image separately from each other, and generates a bidi- 
rection predicted image based on those images. 

For example, a technique wherein an average 
image of the forward direction predicted image and the 
backward direction predicted image is obtained and 
determined as a bidirection predicted image 128 may 
be used. 

Meanwhile, the background predicted image gener- 
ating unit 30 reads out a reference image 1 10 from the 
second frame memory and outputs a background pre- 
dicted image 130 by block matching. 

The prediction mode determining unit 31 inputs 
predicted images selected in the predicted images 127, 
128, 129 and 130 and selects a prediction mode in 
which the difference (prediction error) from the input 
image 101 is minimized. In this instance, a prediction 
mode 126 and a motion vector 123 are outputted from 
the prediction mode determining unit 31 . The prediction 
mode 126 may be determined such that, for example, it 
has a value 0 for the forward direction prediction mode, 
another value 1 for the backward direction prediction 
mode, a further value 2 for the bidirectional prediction 
mode, and a still further value 3 for the background pre- 
diction mode. 

It is to be noted that processing operation of the 
motion vector 123 generated by and outputted from the 



prediction mode determining unit 31 of FIG. 2 is such as 
follows. 

In particular, when searching of reference images is 
performed within a predetermined range and a pre- 

5 dieted image which exhibits a minimum prediction error 
is obtained by each of the predicted image generating 
units, motion vectors 123(a), 123(b), 123(c) and 123(d) 
are outputted from the predicted image generating units 
27 to 30 together with the predicted images, respec- 

10 tively. The outputs are all inputted to the prediction 
mode determining unit 31 , by which one of the predicted 
images 127, 128, 129 and 130 which exhibits a mini- 
mum error from the current image 101 is selected. 
Thus, the motion vector (one of the motion vectors 

w 123(a), 123(b), 123(c) and 123(d)) which provides the 
minimum value is finally outputted as a motion vector 
123 from the prediction mode determining unit 31 . 

FIG. 3 is a block diagram showing an internal con- 
struction of the motion compensation predicting unit 21 . 

20 Referring to FIG. 3, reference numerals 24 and 26 
denote each a selector (switch), and reference numeral 
114 denotes a background predicted image. Subse- 
quently, operation will be described. In the switch 24, 
two switches SW1 and SW2 are opened or closed in 

25 accordance with the determined motion prediction 
mode 126. 

For example, when the prediction mode 126 output- 
ted from the prediction mode determining unit 31 indi- 
cates the bidirection prediction image mode, the switch 

30 SW1 of the selector 24 selects a node B and the switch 
SW2 selects another node C. On the other hand, when 
the background prediction mode is selected, the switch 
SW1 is OFF (provides no selection) and the switch SW2 
selects a further nodeE. 

35 In the former case, the bidirection motion compen- 
sating unit 12 generates a bidirection predicted image 
112 using a motion vector 123. Simultaneously, the out- 
put node from the bidirection motion compensating unit 
12 is selected by the switch 26. Consequently, the bidi- 

40 rection predicted image 1 12 from the motion compensa- 
tion predicting unit 21 is outputted as a determined 
predicted image 115. 

Further, while the embodiment 1 described above 
is constructed such that it includes a motion estimating 

45 unit and a motion compensation predicting unit sepa- 
rately from each other and a prediction mode and a 
motion vector obtained by the motion estimating unit are 
sent to the motion compensation predicting unit so that 
a predicted image is generated by the motion compen- 

50 sation predicting unit, an equivalent function can be 
realized even by such a construction that the two units 
are replaced by a motion estimating/compensating unit 
39 as seen in FIG. 4. 

By the way, in the embodiment 1 described above, 

55 similarly as in the conventional example, coding is per- 
formed in units of a macrobJock which is a processing 
unit for images. 

Meanwhile, in the processing of the MPEG2 of the 
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conventional example described in connection with the 
prior art, three types of pictures including an I picture, a 
P picture and a B picture are involved, and a prediction 
mode is restricted by those pictures. 

In particular, in an I picture, all macroWocks are 
intra coded, and no prediction mode is involved. In a P 
picture, only forward direction prediction is involved, and 
in a B picture, three prediction modes of forward direc- 
tion prediction, backward direction prediction and bidi- 
rection prediction are involved. 

In the meantime, according to the present inven- 
tion, in addition to the pictures described above, two 
other picture types of a PG picture and a PBG picture, 
which will be hereinafter described, are involved. In a 
PG picture, two prediction modes including forward 
direction prediction and background prediction are 
involved, and in a PBG picture, four prediction modes of 
forward direction prediction, backward direction predic- 
tion, bidirection prediction and background prediction 
are involved. 

FIGS. 5A, 5B and 5C show examples of patterns of 
coded pictures. For example, the pattern shown in FIG. 
5A is similar to the conventional example, and similar 
means to that of the prior art may be applied. For the 
pattern shown in FIG. 5B, two prediction modes includ- 
ing background prediction from a background image 
(indicated at "BG" in FIG. 5B) written in the second 
frame memory 10 and forward direction prediction from 
an immediately preceding decoded picture are involved, 
and one of the two prediction modes which provides a 
smaller prediction error is selected. 

This operation is performed up to the sixth picture, 
and then beginning with the seventh picture, the picture 
structure changes to the structure of P, B, B, P, ... In this 
instance, up to the sixth picture, a background image is 
recorded in the second frame memory 10. Thereafter, 
however, the ninth picture is first forward direction pre- 
dicted referring to the sixth picture. 

Then, similarly as in the conventional example, the 
seventh and eighth pictures are predicted referring to 
decoded pictures of the sixth picture and the ninth pic- 
ture. 

In FIG. 5B, a dotted line extending from the second 
picture to the "BG" signifies that, for example, contents 
of the decoded image of the second picture are written 
as a background image into the second frame memory. 

As the writing timing, writing may be performed 
after each certain interval of time or in response to a 
control signal from the outside. However, the pattern 
described above is a mere example, and any other pat- 
tern may be available. 

FIG. 5C shows a pattern wherein the first picture is 
an I picture, and it can be seen that a coded picture of 
the I picture is written as a background image into the 
second frame memory. 

Then, the prediction modes of macroblocks of all 
pictures beginning with the third picture are selected 
either to background image prediction or to forward 



direction prediction. This is effective where the back- 
ground image is stationary, and is very effective with a 
scene wherein some person speaks in front of the back- 
ground image since a phenomenon called occlusion 

s wherein the background image comes into and out of 
sight by a movement of the person occurs. Further, 
when the background image is a still picture and is 
known in advance, the background image may be writ- 
ten into the second frame in advance before coding 

10 processing is started. 

It is to be noted that the pattern of coded pictures 
may take any pattern other than those shown in FIGS. 
5A,5Band5C. 

Subsequently, operation of the frame memory 

15 selecting unit 35 shown in FIG. 1 will be described. 

In the frame memory selecting unit 35, it is deter- 
mined into which one of the first frame memory 9 and 
the second frame memory 10 the locally decoded 
image 108 is to be written. As the determination 

20 method, a technique may be employed wherein, for 
example, as seen from another construction of the cod- 
ing apparatus of the embodiment 1 shown in FIG. 6, a 
control signal 140 from the frame re-arranging unit 1 is 
received by the frame memory selecting unit 35 and 

25 switching between the first frame memory 9 and the 
second frame memory 10 is performed in accordance 
with the received control signal 140 by the frame mem- 
ory selecting unit 35. 

In this instance, since the types of a currently coded 

30 picture and another picture to be coded subsequently 
are known, for example, a decoded image is written into 
the first frame memory 9 till the "BG" end indicated in 
FIG. 5B unless a signal from the outside is received, 
and since the picture structure thereafter changes to the 

35 structure of P, B, B, P, . . . , the frame memory for a subject 
of writing should be selected adaptively as in the con- 
ventional example. 

Further, as seen in FIG. 5B, as writing of a back- 
ground image from a decoded picture at a certain posi- 

40 tion into the second frame memory 10, for example, 
when a scene change is detected, the decoded image 
may be written after a predetermined interval of time 
since then. 

For the detection method for a scene change, a 
45 technique conventionally used may be used. For exam- 
ple, a method wherein, if the number of those of mac- 
roblocks in one frame with which the prediction error is 
higher than a threshold value is larger than a certain 
value, this is regarded as a scene change. 
so It is a matter of course that various other techniques 
than that described just above are available. 

Further, while, in the image coding apparatus of the 
present embodiment 1, the first and second frame 
memories are provided as storage means to realize a 
55 construction for switching of motion compensation pre- 
diction, for implementation of the hardware, a plurality of 
frame memories can be provided at a time by cutting a 
memory having a storage capacity for the plurality of 
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frame memories based on internal addresses. 

As described above, with the image coding appara- 
tus of the present embodiment 1, since a background 
image is stored and motion compensating prediction is 
performed using background prediction based on the 
background image, coding can be performed while 
keeping a high prediction efficiency without being influ- 
enced by a coding sequence. 

It is to be noted that, while storage control of a 
background image into the frame memories is 
described in the foregoing description, it is a matter of 
course that the background image here signifies an 
image which is stored continuously and does not signify 
contents themselves of an image. 

In particular, since images which are successively 
updated like a conventional picture array include some 
image which is effective for later prediction, this image is 
continuously stored independently of storage by the 
updating procedure, and here, this image is referred to 
as background image. 

Embodiment 2. 

FIG. 7 is a block diagram of a decoding apparatus 
for moving pictures according to an embodiment 2 of 
the present invention. Referring to FIG. 7, reference 
numeral 23 denotes a motion compensating unit, and 
46 a demultiplexing unit. The other components than 
those are similar to those employed in the embodiment 
1 , and accordingly, repetitive description of them is omit- 
ted here. 

Operation will be described subsequently. 

The decoding apparatus of the present embodi- 
ment 2 corresponds to the coding apparatus described 
in connection with the embodiment 1, and a basic 
processing procedure for decoding thereof is similar to 
that of the decoding apparatus described in the conven- 
tional example described in the prior art. Thus, descrip- 
tion will be given here principally of differences between 
them. 

A locally decoded image 108 is inputted to the 
frame memory selecting unit 35. The frame memory 
selecting unit 35 receives the locally decoded image 
1 08, selects a frame memory of a subject of writing, and 
transfers a selected decoded image 134 or 135 to the 
first frame memory 9 and the second frame memory 1 0. 

Then, the decoded image is written into the first 
frame memory 9 or the second frame memory 10. 

Meanwhile, the motion compensating unit 23 reads 
out reference images 109 and 1 10 from the two frame 
memories and generates a predicted image 115 in 
accordance with a predetermined motion prediction 
mode 126 in a similar procedure to that in local decod- 
ing of the coding apparatus. 

FIG. 8 is a block diagram showing an internal con- 
struction of the motion compensating unit 23. Referring 
to FIG. 8, reference numeral 32 denotes a switch. 

Subsequently, operation will be described. 



One of the predicted image generating units 27 to 
30 which corresponds to a selected prediction mode 
126 reads out reference images 109 or 1 10 to generate 
a predicted image. Further, the switch 32 is switched in 
s response to the selected prediction mode so that a 
finally determined predicted image 1 15 is outpurted. 

Embodiment 3. 

10 FIG. 9 is a block diagram of an image coding appa- 
ratus according to an embodiment 3 of the present 
invention. Referring to FIG. 9, reference numeral 33 
denotes a motion compensation predicting unit, 34 a 
third frame memory, 37 a frame memory selecting unit, 

15 41 a motion estimating unit, 133 a reference image of 
the third frame memory, and 136 a selected locally 
decoded image. The other components than those 
mentioned above are similar to those employed in the 
embodiment 1 , and accordingly, repetitive description of 

20 them is omitted here. 

The image coding apparatus of the present embod- 
iment 3 is characterized in that it includes the third 
frame memory in addition to the construction of the 
image encoder of the embodiment 1 shown in FIG. 1. 

25 Subsequently, operation will be described. 

Reference images 109, 110 and 133 are read out 
from the three frame memories 9, 10 and 34 in which 
coded images in the past are stored, and motion predic- 
tion is performed by the motion estimating unit 41 . A 

30 motion vector 1 23 and a prediction mode 1 26 obtained 
by the motion prediction are inputted to the motion com- 
pensation predicting unit 33. 

The motion compensation predicting unit 33 selects 
a reference image necessary for generation of a prede- 

35 termined motion predicted image from among the refer- 
ence images 109, 110 and 133 based on the 
determined prediction mode 126, and outputs the deter- 
mined predicted image 115. 

Meanwhile, a locally decoded image 108 is written, 

40 after it is determined by the frame memory selecting 
unit 37 into which frame memory the locally decoded 
image 1 08 should be written, as a reference image 1 34, 
1 35 or 1 36 into the thus determined frame memory. 
FIG. 10 shows an internal construction of the 

45 motion estimating unit 41. Referring to FIG. 10, refer- 
ence numeral 42 denotes a prediction mode determin- 
ing unit. 

The motion estimating unit 41 shown in FIG. 10 has 
a construction which includes, in addition to the motion 

so estimating unit 15 shown in FIG. 2, a background pre- 
dicted image generating unit 30 for inputting a reference 
image 133 from the third frame memory. 

The forward direction predicted image generating 
unit 27 inputs an input image 101 and a reference 

55 image 1 09 of the first frame memory and outputs a for- 
ward direction predicted image 127, and the bidirection 
predicted image generating unit 28 inputs the input 
image 101, the reference image 109 of the first frame 
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memory and a reference image 110 of the second 
frame memory and outputs a bidirection predicted 
image 128. 

The backward direction predicted image generating 
unit 29 inputs the input image 101 and the reference 
image 1 10 of the second frame memory and outputs a 
backward direction predicted image 129, and the back- 
ground predicted image generating unit 30 inputs the 
input image 101 and a reference image 133 of the third 
frame memory and outputs a background predicted 
image 130. 

The prediction mode determining unit 42 calculates 
absolute value differences between predicted images 
27, 28, 29 and 30 mentioned above and input the image 
101, determines a prediction mode which exhibits a 
minimum one of the absolute value differences, and out- 
puts the determined prediction mode as a prediction 
mode 126. Simultaneously, the prediction mode deter- 
mining unit 42 outputs a motion vector 123. 

FIG. 1 1 is a block diagram of an internal construc- 
tion of the motion compensation predicting unit 33. 
Referring to FIG. 1 1 , a switch 25 is opened or closed in 
response to the prediction mode 126 so that the refer- 
ence image 109 or 1 10 is inputted to a selected one of 
the motion compensating units. For example, when the 
forward direction prediction mode is selected, a switch 
SW1 is switched to a node A while another SW2 is 
switched off. However, when the bidirection prediction 
mode is selected, the switch SW1 is switched to another 
node B while the switch SW2 is switched to a further 
node C. 

When the background prediction mode is selected, 
a reference image 133 is inputted directly and referred 
to. Subsequently, in the switch 26, the switches SW1 
and SW2 are switched to nodes corresponding to the 
prediction mode 126, and a predicted image 115 deter- 
mined finally is outputted from the switch 26. 

Further, while, in the present embodiment 3, the 
first, second and third frame memories are provided to 
realize a construction for switching of motion compen- 
sating prediction, for implementation of the hardware, a 
plurality of frame memories can be provided at a time by 
cutting a memory having a storage capacity for the plu- 
rality of frame memories based on internal addresses. 

FIGS. 12A, 12B and 12C are diagrammatic views 
illustrating re-writjng operation of the frame memories in 
the present embodiment 3, and in the following, the re- 
writing operation will be described including a relation- 
ship to the operation of the frame memory selecting unit 
37 described hereinabove with reference to FIGS. 6A, 
6B and 6C. 

FIGS. 12A, 12B and 12C show three different pat- 
terns. In FIG. 12A, PG pictures of background predic- 
tion and forward direction prediction appear beginning 
with the sixth picture, and the construction continues up 
to the ninth picture. Thereafter, the structure of IBBP is 
restored beginning with the 10th picture. 

In FIG. 12B, switching among all prediction modes 



of forward direction prediction, backward direction pre- 
diction, bidirection prediction and background prediction 
is possible with the first, second, fourth, fifth, seventh, 
eighth, tenth and eleventh pictures, and the prediction 

5 efficiency is highest. Further, also in this instance, while 
writing as a background image into the third frame 
memory is enabled at any time, in the example of FIG. 
12B, writing into the third frame memory for a back- 
ground image is performed from the fifth and tenth pic- 

10 tures. 

In FIG. 12C, PG pictures of background prediction 
and forward direction prediction appear with the third, 
sixth, ninth and twelfth pictures. 

In those operations, since it is already known of 

15 which picture type a currently decoded picture is, a 
frame memory into which the locally decoded image 
108 is to be written is determined by itself in accordance 
with the picture type by the frame memory selecting unit 
37. In particular, where the pattern has the structure of 

20 IBBP, for the I picture, the locally decoded image 1 08 is 
written into the first frame memory, but, for the P picture, 
the locally decoded image 1 08 is written into the second 
frame memory. For the B pictures, the locally decoded 
image 108 is written into none of the frame memories. 

25 It is to be noted that, as described already, a certain 
decoded image is written as a background image also 
into the third frame after a certain interval of time or in 
response to a control signal from the outside. 

30 Embodiment 4. 

FIG. 1 3 is a block diagram of a decoding apparatus 
for moving pictures according to an embodiment 4 of 
the present invention. The decoding apparatus corre- 

35 sponds to the coding apparatus of the embodiment 3 
shown in FIG. 9. Referring to FIG. 13, reference 
numeral 36 denotes a motion compensating unit. Of the 
other components than that just mentioned, those com- 
ponents denoted by the same reference numerals to 

40 those used in the embodiments 1 to 3 are similar ele- 
ments, and accordingly, repetitive description of them is 
omitted here. 

Subsequently, operation will be described. 

The motion compensating unit 36 performs motion 

45 compensation referring to reference images 109, 110 
and 133 read out from the first frame memory 9, the 
second frame memory 10 and the third frame memory 
1 1 and outputs a predicted image 1 15. 

Decoded images are re-arranged by the displayed 

so frame re-arranging unit 38 again such that they appear 
in order of time for displaying, and an output image 137 
is obtained as a result of the re-arrangement. 

FIG. 14 is a block diagram showing an internal con- 
struction of the motion compensating unit 36. Referring 

55 to FIG. 14, one of predicted images generated by the 
individual predicted image generating units is selected 
in response to a prediction mode 126 by the switch 32. 
Then, the selected predicted image 1 15 is outputted to 
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the adding unit 8. 

The fourth embodiment presents similar effects to 
those of the imaging coding apparatus of the embodi- 
ment 3. 

Embodiment 5. 

While the image decoding apparatus of the embod- 
iment 4 described above performs re-writing to a back- 
ground image illustrated in FIGS. 5B and 5C in units of 
a picture, it is sometimes possible to perform prediction 
efficiently if writing to a background image is performed 
in units of a macroblock. 

As the technique for re-writing to a background 
image, for example, a technique wherein updating is 
performed after each predetermined interval of time in 
coding processing or another technique wherein, when 
all pixels in a macroblock at a certain position are not 
referred to for prediction for more than a certain period 
of time, a control signal is generated to re-write only the 
macroblock in a background image with a decoded 
image may be used. 

FIG. 15 illustrates this. Referring to FIG. 15, at a 
timing for writing from the second picture of FIG. 5B into 
the background image "BG", only a macroblock in a 
region of slanted lines in FIG. 15 is written as it is into 
the second frame memory and is used as part of a ref- 
erence image for direction of the third picture. 

Similarly, also where the image coding apparatus of 
the embodiment 5 described above includes three 
frame memories, re-writing into a background image 
shown in FIGS. 12B and 12C is performed in units of a 
macroblock. As the technique for re-writing, the same 
operation as described above may be performed. 

As described above, since re-writing of contents of 
an image in each frame memory is performed in units of 
a macroblock after each certain interval of time or in 
response to a control signal from the outside, the con- 
tents of the image in the frame memory can always be 
kept, at a finer level, to contents from which a high pre- 
diction efficiency for background prediction can be 
obtained. 

Embodiment 6. 

Also an image decoding apparatus which corre- 
sponds to the image coding apparatus which includes 
three frame memories described above in connection 
with the image coding apparatus of the embodiment 5 
can perform re-writing to the background image in units 
of a macroblock. 

For example, in an image decoding apparatus 
shown in FIG. 7, after a decoded image 108 is selected 
by the frame memory selecting unit 35, a macroblock of 
the background image at the same position as that of 
the macroblock mentioned above is re-written to a 
selected decoded image 135. It is to be noted that the 
updating in units of a macroblock may be performed 



after a certain interval of time or in response to a control 
signal from the outside. 

Similarly, also in the decoding apparatus shown in 
FIG. 13, re-writing to a background image illustrated in 
5 FIGS. 12B and 12C is performed in units of a macrob- 
lock. As the technique for re-writing, the same operation 
as described above may be performed. 

Embodiment 7. 

10 

It is also effective to vary the motion searching 
range upon background prediction by the motion esti- 
mating unit 15 of the image coding apparatus of the 
embodiment 1 shown in FIG. 1 or the motion compen- 

is sation predicting unit 33 shown in FIG. 3 to the search- 
ing range of forward direction prediction or backward 
direction prediction. 

To this end, it is advisable to set, making use of the 
fact that, for example, background prediction acts effec- 

20 tively when the motion vector from the background is 0, 
the searching range to a smaller range than that for any 
other prediction. 

The image coding apparatus of the present embod- 
iment 7 exhibits an additional effect in that the searching 

es time is reduced and that, since codes obtained by vari- 
able length coding of motion vectors can be set compar- 
atively short, the coding information amount of motion 
vectors can be reduced,. 

30 Embodiment 8. 

FIG. 16 is a block diagram of an image coding 
apparatus according to an embodiment 8 of the present 
invention. Referring to FIG. 16, reference numeral 47 
35 denotes a differential vector generating unit, and 141 a 
difference vector. Of the other components than those 
mentioned above, those components denoted by the 
same reference numerals to those used in the embodi- 
ment 1 are similar elements, and accordingly, repetitive 
*o description of them is omitted here. 

The differential vector generating unit 47 calculates 
a difference vector 141 between a current motion vector 
123 and a reference vector. Then, the difference vector 
141 is variable length coded by the variable length cod- 
45 ingunit17. 

FIGS. 17A and 17B illustrates a coding method for 
a motion vector, and particularly, FIG. 17A shows a ref- 
erence motion vector for the first frame memory 9 and 
FIG. 17B shows a reference motion vector for the see- 
so ond frame memory 10. 

Operation will be described subsequently. 
Referring to FIGS. 17A and 17B, each rectangular 
frame denotes one macroblock. It is known that, for a 
motion vector MV(1) of a current macroblock obtained 
55 by reading out a reference image in the first frame mem- 
ory and performing motion compensating prediction of 
the reference image, it is effective to actually variable 
length code, using three motion vectors MV1(1), 
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MV2(1) and MV3(1) of already coded and decoded 
macroblocks as candidate vectors, difference values of 
the motion vector MV(1) from them. 

For example, if it is tried to use a median of the 
motion vectors MV1(1), MV2(1) and MV3(1) as a candi- 
date vector, then the difference vector PMV(1) can be 
represented by the following expression: 

PMV(1) = MV(1) - median (MV1(1), MV2(1), 
MV3(1)) where "median" is an operator for calcula- 
tion of a median. 

Similarly, for the second frame memory, 

PMV(2) = MV(2) - median (MV1(2), MV2(2), 
MV3(2)) 

FIG. 18 is a block diagram of an image coding 
apparatus which includes a differential vector generat- 
ing unit 47 in addition to the construction of the image 
coding apparatus shown in FIG. 9. 

For calculation of a difference vector, in addition to 
the operation described above, a reference motion vec- 
tor PMV(3) for the third frame memory may be calcu- 
lated and variable length coded. 

The information generation amount of motion vec- 
tors can be supplied in such a manner as described 
above. 

Embodiment 9. 

FIGS. 19 and 20 are block diagrams of decoding 
apparatus which correspond to the coding apparatus of 
the embodiment 8 described hereinabove with refer- 
ence to FIGS. 16 and 18 in which a difference vector is 
used, respectively. Referring to FIGS. 19 and 20, refer- 
ence numeral 48 denotes a motion vector adding unit. 
The other components are similar to those of the decod- 
ing apparatus of the embodiment 2 shown in FIG. 7, and 
accordingly, repetitive description of them is omitted 
here. 

In the decoding apparatus of the present embodi- 
ment 9, a difference vector 141 variable length decoded 
by the variable length decoding unit 22 is added to a ref- 
erence vector by the variable length decoding unit 22 to 
calculate a motion vector 123. 

The processing following it is the same as the oper- 
ation of the decoding apparatus of the embodiment 2 
shown in FIG. 7, and therefore, repetitive description of 
it is omitted here. 

Embodiment 10. 

While, in the coding apparatus of FIG. 1, an entire 
screen in a picture is used as a subject of coding, the 
image coding apparatus of the present embodiment 10 
is constructed such that the picture type for coding is 
variable in units of one of a plurality of subject images 
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(objects) which construct the screen. 

Referring to FIG. 21, for example, if a screen is 
composed of an object 1 (fish), an object 2 (water: back- 
ground picture) and an object 3 (ball) and boundaries 
5 among them are known, then those objects can be 
coded using different techniques from one another. 

In the image coding apparatus of the present 
embodiment 10, such coding techniques are realized by 
using different picture types from one another. For 
10 example, since the object 1 exhibits a comparatively 
large amount of motion, the construction of picture 
types of FIG. 5A is used for the object 1 taking it into 
consideration that bidirection prediction is higher in pre- 
diction efficiency than background prediction. 
is On the other hand, since the object 2 is an image 
which exhibits little motion, it is effective to use back- 
ground prediction for it. Accordingly, the construction of 
FIG. 5C should be used. However, if such a variation 
that a scene changes rapidly occurs with a certain midst 
20 picture, then the construction which includes B pictures 
beginning with the midst picture as seen in FIG. 5B 
should be employed. 

FIG. 22 is a block diagram showing a concrete 
example of the image coding apparatus provided by the 
25 present embodiment 1 0. Referring to FIG. 22, reference 
numeral 42 denotes an object distinguishing unit, 43 a 
first frame memory group, 44 a second frame memory 
group, and 138 an object identification signal. 

Operation will be described subsequently. 
30 An input image 100 includes identification signals 
applied to individual objects in advance, and the identi- 
fication signals are identified by the object distinguish- 
ing unit 42. The number of each of the thus identified 
objects is outputted as an object identification signal 
35 1 38 from the object distinguishing unit 42. 

The motion estimating unit 15 selects, from among 
the first frame memory group 43 and the second frame 
memory group 44, a frame memory which corresponds 
to the object of the subject of coding in accordance with 
40 the object identification signal 138, reads out a refer- 
ence image from the selected frame memory and per- 
forms motion prediction. 

Meanwhile, the motion compensation predicting 
unit 21 selects a frame memory corresponding to a pre- 
45 determined object in accordance with a motion predic- 
tion mode 126 determined by the motion estimating unit 
15 and generates a predicted image 115. 

On the other hand, the frame memory selecting unit 
35 writes a decoded image 108 into one of the frame 
so memories of a predetermined one of the frame memory 
groups which corresponds to a predetermined object in 
accordance with the object identification signal 138. 

Further, the object identification signal 138 is multi- 
plexed together with other coding information by the 
55 multiplexing unit 45 and sent out as a multiplexed bit 
stream 139 to an external apparatus (not shown). 

While, in the image coding apparatus of the present 
embodiment 10, the first and second memory groups 
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are provided to realize the construction for switching of 
motion compensating prediction, for implementation of 
the hardware, a plurality of frame memories can be pro- 
vided at a time by cutting a memory having a storage 
capacity for the plurality of frame memories based on 
internal addresses. As described above, with the image 
coding apparatus of the present embodiment 10, since 
a prediction structure which conforms with motion of an 
object can be taken, the overall prediction efficiency is 
improved. 

Embodiment 1 1 . 

A block diagram of an image decoding apparatus 
which corresponds to the image coding apparatus of the 
embodiment 10 shown in FIG. 22 is shown in FIG. 23. 
Referring to FIG. 23, reference numeral 46 denotes a 
demultiplexing unit. The other components are similar 
to those of the image decoding apparatus of, for exam- 
ple, the embodiment 4 shown in FIG. 13, and accord- 
ingly, repetitive description of them is omitted here. 

Operation will be described subsequently. 

In response to an object identification signal 138 
demultiplexed by the demultiplexing unit 46, the motion 
compensating unit 23 reads out a reference image from 
one of frame memories of a predetermined frame mem- 
ory group which corresponds to a predetermined object, 
and performs motion compensation corresponding to a 
prediction mode to generate a predicted image 115. 

In the meantime, the frame memory selecting unit 
35 writes a decoded image 108 into one of the frame 
memories of a predetermined frame memory group 
which corresponds to a predetermined object in accord- 
ance with the object identification signal 138. The other 
processing is similar to that of the image decoding 
apparatus of the embodiment 4 shown in FIG. 13, and 
accordingly, repetitive description of it is omitted here. 

Embodiment 12. 

FIG. 24 is a block diagram of an image coding 
apparatus which includes a further frame memory 
group in addition to the construction of the embodiment 
10 described hereinabove with reference to FIG. 22 
such that it may include totaling three frame memory 
groups. Referring to FIG. 24, reference numeral 49 
denotes a third frame memory group. The other compo- 
nents are similar to those of the image coding apparatus 
of the embodiment 10 shown in FIG. 22, and accord- 
ingly, repetitive description of them is omitted here. 

Subsequently, operation will be described. 

An input image 100 includes identification signals 
applied to individual objects in advance, and the identi- 
fication signals are identified by the object distinguish- 
ing unit 42. The number of each of the thus identified 
objects is outputted as an object identification signal 
138 from the object distinguishing unit 42. 

The motion estimating unit 15 selects, from among 



the first frame memory group 43, the second frame 
memory group 44 and the third frame memory group 
49, a frame memory which corresponds to the object of 
the subject of coding in accordance with the object iden- 
s tification signal 138, reads out a reference image from 
the selected frame memory and performs motion pre- 
diction. 

Meanwhile, the motion compensation predicting 
unit 21 selects a frame memory corresponding to a pre- 

10 determined object in accordance with a motion predic- 
tion mode 1 26 determined by the motion estimating unit 
15 and generates a predicted image 115. 

On the other hand, the frame memory selecting unit 
35 writes a decoded image 108 into one of the frame 

15 memories of a predetermined one of the frame memory 
groups which corresponds to a predetermined object in 
accordance with the object identification signal 138. 
Further, the object identification signal 138 is multi- 
plexed together with other coding information by the 

w multiplexing unit 45 and sent out as a multiplexed bit 
stream 139. 

While, in the image coding apparatus of the present 
embodiment 12, the first, second and third memory 
groups are provided to realize the construction for 
25 switching of motion compensating prediction, for imple- 
mentation of the hardware, a plurality of frame memo- 
ries can be provided at a time by cutting a memory 
having a storage capacity for the plurality of frame mem- 
ories based on internal addresses. 

30 

Embodiment 13. 

A block diagram of an image decoding apparatus 
corresponding to the image coding apparatus of the 

35 embodiment 12 shown in FIG. 24 is shown in FIG. 25. 
Referring to FIG. 25, reference numeral 49 denotes a 
third frame memory group. The other components are 
similar to those of the image decoding apparatus of. for 
example, the embodiment 11 shown in FIG. 23, and 

40 accordingly, repetitive description of them is omitted 
here. 

Operation will be described subsequently. 
In response to an object identification signal 138 
demultiplexed by the demultiplexing unit 46, the motion 

45 compensating unit 23 reads out a reference image from 
one of frame memories of a predetermined frame mem- 
ory group which corresponds to a predetermined object, 
and performs motion compensation corresponding to a 
prediction mode to generate a predicted image 115. 

so In the meantime, the frame memory selecting unit 
35 writes a decoded image 108 into one of the frame 
memories of a predetermined frame memory group 
which corresponds to a predetermined object in accord- 
ance with the object identification signal 138. 

55 The other processing is similar to that of the image 
decoding apparatus of the embodiment 11 shown in 
FIG. 23, and accordingly, repetitive description of it is 
omitted here. 
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Embodiment 14. 

The image coding apparatus of the embodiment 12 
shown in FIG. 24 may be modified such that re-writing 
of image contents of a region, in which an object of a 
subject of coding is included, of a frame memory corre- 
sponding to the object in the second frame memory in 
which a decoded image of the object in the past is 
stored is performed after each certain interval of time or 
in response to a control signal from the outside. 

FIG. 26 is a diagrammatic view illustrating that, for 
example, with a decoded image of all macroblocks 
including a region occupied by a certain object, image 
contents in a macroblock or macroblocks at the same 
position of a frame memory in the second frame mem- 
ory group which corresponds to the object are re-writ- 
ten. Accordingly, in the case of FIG. 26, contents of 
totaling four macroblocks in two vertical columns and 
two horizontal rows are updated. 

Further, where re-writing of image contents of a 
region, in which an object of a subject of coding is 
included, of a frame memory corresponding to the 
object in the third frame memory in which a decoded 
image of the object in the past is stored is performed 
after each certain interval of time or in response to a 
control signal from the outside, the writing operation into 
a frame memory in the second frame memory group in 
the foregoing description should be applied to the writ- 
ing operation into a frame memory in the third frame 
memory group. 

Also with a decoding apparatus which corresponds 
to the image coding apparatus of the embodiment 12 
shown in FIG. 24 as described above, re-writing of 
image contents of a region, in which an object is 
included, of a frame memory corresponding to the 
object in the second frame memory group in which a 
decoded image of the object in the past is stored can be 
controllably performed after a certain interval of time or 
in response to a control signal from the outside. 

Embodiment 15. 

The image coding apparatus of the embodiment 10 
shown in FIG. 22 can be modified such that searching 
ranges of motion vector searching for a reference image 
from a frame memory of the first frame memory group 
which corresponds to an object and another reference 
image from another frame memory of the second frame 
memory group which corresponds to another object are 
varied for the individual objects. 

For example, in the image coding apparatus of the 
embodiment 10 shown in FIG. 22, if a background 
which exhibits a comparatively small amount of motion 
as an object is stored in advance in a frame memory of 
the second frame memory group which corresponds to 
the object whereas an operation of successively writing 
a decoded image of another object which exhibits a 
comparatively large amount of motion at any time into 



another frame memory of the first frame memory group 
which corresponds to the object is performed, then a 
high prediction efficiency can be maintained for both of 
the objects. 

s Further, the image coding apparatus of the embod- 
iment 12 shown in FIG. 24 may be modified such that 
searching ranges of motion vector searching for a refer- 
ence image from s from memory of the first frame mem- 
ory group which corresponds to an object, another 

10 reference image from another frame memory of the sec- 
ond frame memory group which corresponds to another 
object and a further reference image from a further 
frame memory of the third frame memory group which 
corresponds to a further object are varied for the individ- 

15 ual objects. 

For example, in the image coding apparatus of the 
embodiment 12 shown in FIG. 24, if a background 
which exhibits a comparatively small amount of motion 
as an object is stored in advance in a frame memory of 

20 the third frame memory group which corresponds to the 
object whereas an operation of successively writing a 
decoded image of another object which exhibits a com- 
paratively large amount of motion at any time into 
another frame memory of the first frame memory group 

25 or the second frame memory group which corresponds 
to the object is performed, then a high prediction effi- 
ciency can be maintained for all of the three objects. 

As described above, since searching ranges for a 
motion vector are set separately from each other for a 

30 plurality of frame memory groups referred to by objects, 
for example, for an object which exhibits a compara- 
tively small amount of motion, the information genera- 
tion amount of motion vectors can be reduced by 
making the searching range for a motion vector narrow. 

35 

Embodiment 16. 

FIG. 27 is a block diagram showing an image cod- 
ing apparatus according to an embodiment 16 of the 

40 present invention. Referring to FIG. 27, reference 47 
denotes a differential vector generating unit. The differ- 
ential vector generating unit 47 holds motion vectors in 
the past obtained by referring to images of individual 
objects from frame memories of the first frame memory 

45 group which correspond to the objects and motion vec- 
tors in the past obtained by referring to images of the 
individual objects from frame memories of the second 
frame memory group which correspond to the objects in 
the image coding apparatus of the embodiment 10 

so shown in FIG. 22 separately for certain periods of time 
and calculates difference vectors separately for the indi- 
vidual objects. The other construction is similar to that of 
the image coding apparatus of the embodiment 10 
shown in FIG. 22, and accordingly, repetitive description 

55 of it is omitted here. 

Subsequently, operation will be described. 
The motion estimating unit 15 performs motion esti- 
mation of a current image 101 of an object of a subject 



14 



27 



EP 0 817 491 A2 



of coding using an image in a frame memory corre- 
sponding to the object in one of the first frame memory 
group and the second frame memory group selected by 
motion estimation as a reference image to detect a 
motion vector 123. 

Based on the motion vector 123, the differential 
vector generating unit 47 selects a candidate vector 
(MV1, MV2 or MV3 mentioned hereinabove) from 
among motion vectors of the object in the past stored in 
the differential vector generating unit 47 and outputs a 
difference vector 141 of the candidate vector from the 
motion vector 123. The difference vector 141 is coded 
into a variable length codeword by the variable length 
coding unit 17. Accordingly, the differential vector gen- 
erating unit 47 has a memory function of holding motion 
vectors in the past separately for certain periods of time 
for the individual frame memory groups. 

Embodiment 17. 

A block diagram of a decoding apparatus corre- 
sponding to the image coding apparatus of the embodi- 
ment 16 shown in FIG. 27 is shown in FIG. 28. Referring 
to FIG. 28, reference numeral 48 denotes a motion vec- 
tor adding unit which selects a candidate vector from 
among motion vectors of an object in the past stored in 
advance therein and adds the selected candidate vector 
to a difference vector 141 variable length decoded by 
the variable length decoding unit 22. The other con- 
struction is similar to that of the image coding apparatus 
of the embodiment 10 shown in FIG. 22, and accord- 
ingly, repetitive description of it is omitted here. 

Subsequently, operation will be described. 

In the image decoding apparatus of the present 
embodiment 17, a difference vector 141 variable length 
coded by the variable length decoding unit 22 is sup- 
plied to the motion vector adding unit 48, by which a 
candidate vector is selected from among motion vectors 
of an object in the past stored therein and added to the 
difference vector 1 41 to regenerate a motion vector 1 23. 

The motion vector 123 is sent to the motion com- 
pensating unit 23. The motion compensating unit 23 
receives the motion vector 123, reads out an image in 
the memory group 43 or 44 corresponding to the object 
in the frame memory group selected by the frame mem- 
ory selecting unit 35 as a reference image, and outputs 
a predicted image 115. The other processing is similar 
to the operation of the image decoding apparatus of the 
embodiment 11 shown in FIG. 23, and accordingly, 
repetitive description of it is omitted here. 

Embodiment 18. 

A construction of an image coding apparatus which 
includes a third frame memory group 49 in addition to 
the construction of the image coding apparatus of the 
embodiment 16 shown in FIG. 27 is shown in FIG. 29. 
The other construction is similar to that of the image 



coding apparatus of the embodiment 16 shown in FIG. 
27, and accordingly, repetitive description of it is omitted 
here. 

Subsequently, operation will be described. 

5 The motion estimating unit 1 5 performs motion esti- 
mation of a current image 101 of an object of a subject 
of coding using an image in a frame memory corre- 
sponding to the object in one of the first frame memory 
group, the second frame memory group and the third 

10 frame memory group selected by motion estimation as 
a reference image to detect a motion vector 123. 

Based on the motion vector 123, the differential 
vector generating unit 47 selects a candidate vector 
(MV1, MV2 or MV3 mentioned hereinabove) from 

w among motion vectors of the object in the past stored in 
the differential vector generating unit 47 and outputs a 
difference vector 141 of the candidate vector from the 
motion vector 123. The difference vector 141 is coded 
into a variable length codeword by the variable length 

20 coding unit 17. 

Also in this instance, the differential vector generat- 
ing unit 47 has a memory function of holding motion 
vectors in the past separately for certain periods of time 
for the individual frame memory groups. Since the other 

25 processing is similar to the operation of the image 
decoding apparatus of the embodiment 1 1 shown in 
FIG. 23, repetitive description of it is omitted herein. 

Embodiment 19. 

30 

A construction of an image decoding apparatus cor- 
responding to the image coding apparatus of the 
embodiment 18 shown in FIG. 29 is shown in FIG. 30. 
Referring to FIG. 30, reference numeral 49 denotes a 

35 third frame memory group. Since the other construction 
is similar to that of the image decoding apparatus of the 
embodiment 17 shown in FIG. 28, repetitive description 
of it is omitted here. 

Subsequently, operation will be described. 

40 A difference vector 1 41 variable length coded by the 
variable length decoding unit 22 is supplied to the 
motion vector adding unit 48, by which a candidate vec- 
tor is selected from among motion vectors of an object 
in the past stored therein and added to the difference 

45 vector 141 to regenerate a motion vector 123. The 
motion vector 123 is sent to the motion compensating 
unit 23. The motion compensating unit 23 reads out a 
reference image in a frame memory corresponding to 
the object in the selected frame memory group, and out- 

so puts a predicted image 1 1 5. 

As described above, if a differential vector generat- 
ing unit which has a memory function of holding a 
number of motion vectors, which is equal to the number 
of the frame memory groups, in the past separately for 

55 certain periods of time for the individual frame memory 
groups and calculates a difference vector between a 
detected motion vector and a candidate vector is pro- 
vided, then the information generation amount of motion 
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vectors can be suppressed. 

As described above, with the image coding appara- 
tus of the present invention, since a background image 
is stored and motion compensating prediction is per- 
formed using background prediction based on the 5 
stored background image, there is an effect that coding 
can be performed while keeping a high prediction effi- 
ciency without being influenced by a coding sequence. 

Further, with the image coding apparatus and the 
image decoding apparatus of the present invention, 10 
since re-writing of image contents in the individual 
frame memories is performed in units of a picture after 
a certain interval of time or in response to a control sig- 
nal from the outside, there is another effect that the 
image contents of the frame memories can always be is 
kept to contents with which a high prediction efficiency 
in background prediction can be obtained. 

Further, with the image coding apparatus and the 
image decoding apparatus of the present invention, 
since re-writing of the image contents of the individual 20 
frame memories is performed in units of a macroblock 
after a certain interval of time or in response to a control 
signal from the outside, there is a further effect that the 
image contents of the frame memories can always be 
kept to contents with which a high prediction efficiency zs 
in background prediction can be obtained with a finer 
level. 

Further, with the image coding apparatus and the 
image decoding apparatus of the present invention, 
since the searching ranges for a motion vector to be 30 
used for motion estimation are variably set for the plural- 
ity of frame memories provided in the coding apparatus, 
for example, when motion is to be searched for from ref- 
erence to a frame memory in which a screen which 
involves a comparatively small amount of motion is writ- 35 
ten, a comparatively short code can be given, and 
accordingly, there is a still further effect that the coding 
information amount of motion vectors can be reduced. 

Further, with the image coding apparatus and the 
image decoding apparatus of the present invention, 40 
since the differential vector generating unit which has a 
memory function of holding a number of motion vectors, 
which is equal to the number of the frame memories, in 
the past separately for a certain period of time and cal- 
culates a difference vector between a detected motion 45 
vector and a candidate vector is provided, there is a yet 
further effect that the information generation amount of 
motion vectors can be suppressed. 

Further, with the image coding apparatus and the 
image decoding apparatus of the present invention, so 
since motion compensating prediction is performed 
using the plurality of frame memories for the individual 
objects which construct a screen, a prediction structure 
conforming to motion of the objects can be taken, and 
consequently, there is a yet further effect that the overall 55 
prediction efficiency is improved. 

Further, with the image coding apparatus and the 
image decoding apparatus of the present invention, 



since only regions of the frame memories in the frame 
memory groups in which an object of a subject of coding 
is included are re-written after a certain interval of time 
or in response to an external control signal, there is a 
yet further effect that a high efficiency in background 
prediction can be maintained. 

Further, with the image coding apparatus and the 
image decoding apparatus of the present invention, 
since the searching ranges for a motion vector are set 
separately for the plurality of frame memory groups 
referred to by an object, there is a yet further effect that, 
for example, for an object which exhibits a compara- 
tively small amount of motion, the information genera- 
tion amount of motion vectors can be reduced by 
making the searching range for a motion vector narrow. 

Furthermore, with the image coding apparatus and 
the image decoding apparatus of the present invention, 
since the differential vector generating unit which has a 
memory function of holding a number of motion vectors, 
which is equal to the number of the frame memory 
groups, in the past separately for certain periods of time 
for the individual frame memory groups and calculates a 
difference vector between a detected motion vector and 
a candidate vector is provided, there is an additional 
effect that the information generation amount of motion 
vectors can be suppressed. 

Claims 

1 . An image coding apparatus comprising: 

storage means (9, 10) for storing a plurality of 
decoded images; 

motion compensating prediction means (15, 
39) for performing motion compensating pre- 
diction corresponding to an input image based 
on the plurality of decoded images stored in 
said storage means (9, 10) to produce a motion 
vector and for generating a predicted image 
based on the motion compensating prediction; 
prediction error calculation means (2) for calcu- 
lating a difference between the predicted 
image generated by said motion compensating 
prediction means (15, 39) and the input image 
to calculate a prediction error image; 
decoding means (8) for generating the 
decoded images from the prediction error 
image calculated by said prediction error calcu- 
lation means (2) and the predicted image; 
background image storage control means (35) 
for selecting one of the decoded images which 
is to be continuously stored in said storage 
means (9, 1 0) as a background image and stor- 
ing the background image into said storage 
means (9, 10); and 

background motion compensation means (14, 
39) for performing motion compensating pre- 
diction corresponding to the input image based 



16 



31 



EP 0 817 491 A2 



on the background image to generate a motion 
vector and generating a predicted image based 
on the motion compensating prediction. 

2. An image decoding apparatus comprising: 

storage means (9, 10) for storing a plurality of 
decoded images; 

motion compensation means (23) for perform- 
ing motion compensating prediction based on 
the decoded images stored in said storage 
means (9, 10) to generate a motion compen- 
sated image; 

decoding means (8) for generating the coded 
images from the motion compensated image 
from said motion compensation means (23) 
and a prediction error image; 
background image storage control means (35) 
for selecting one of the decoded images which 
is to be continuously stored in said storage 
means (9, 1 0) as a background image and stor- 
ing the background image into said storage 
means (9, 10); and 

background predicted image generation 
means (30) for generating a background pre- 
dicted image based on the background image. 

3. An image coding apparatus as set forth in claim 1 , 
wherein re-writing of image contents into said stor- 
age means (9, 10) by said background image stor- 
age control means (35) is performed in units of a 
picture or in units of a macroblock after a predeter- 
mined interval of time or in response to a control 
signal from the outside. 

4. An image decoding apparatus as set forth in claim 

2, wherein re-writing of image contents into said 
storage means (9, 10) by said background image 
storage control means (35) is performed in units of 
a picture or in units of a macroblock after a prede- 
termined interval of time or in response to a control 
signal from the outside. 

5. An image coding apparatus as claimed in claim 1 or 

3, wherein said background motion compensation 
means (1 4, 39) has a variable searching range for a 
motion vector from the background images. 

6. An image coding apparatus as claimed in claim 2 or 

4, further comprising differential vector generation 
means (47) for holding a motion vector obtained 
from said motion compensating prediction means 
(15, 39) or said background motion compensation 
means (14, 39) and calculating a difference vector 
between the generated motion vector and the 
motion vector in the past, wherein the difference 
vector is variable length coded. 



7. An image decoding apparatus as claimed in claim 
2, 4 or 6, further comprising a motion vector adding 
unit (48) for holding a motion vector decoded in the 
past and adding the motion vector decoded in the 

5 past to a difference vector to regenerate a motion 
vector. 

8. An image coding apparatus which outputs a coded 
bit stream of moving pictures comprising: 

10 

a plurality of frame memory groups (43, 44) for 
storing, individually for a plurality of objects 
which compose a screen, decoded images of 
the objects in the past; 
15 a frame memory selecting unit (35) for select- 

ing, in response to a control signal, into a frame 
memory of which one of said plurality of frame 
memory groups (43, 44) a decoded image is to 
be written; 

20 a motion compensation predicting unit (21) for 

selecting one of forward direction prediction, 
backward direction prediction, bidirection pre- 
diction and background prediction in units of an 
object using reference images read out from 

25 frame memories of said plurality of frame mem- 

ory groups (43, 44) provided for the individual 
objects to perform motion compensating pre- 
diction; 

a subtractor (2) for calculating a difference 
so between the predicted image and a current 

image to calculate a prediction error image; 
an adding unit (8) for adding the predicted 
image from the reference images and the pre- 
diction error image of the current image; and 
35 a variable length coding unit (17) for variable 

length coding information. 

9. An image decoding apparatus which decodes a 
coded bit stream of moving pictures comprising: 

40 

a plurality of frame memory groups (43, 44) for 
storing, individually for a plurality of objects 
which construct a screen, decoded images of 
the objects; 

45 a frame memory selecting unit (35) for select- 

ing, in response to a control signal, into a frame 
memory of which one of said plurality of frame 
memory groups (43, 44) the coded images are 
to be written for the individual objects; 

so a variable length decoding unit (22) for variable 

length decoding the coded bit stream; and 
a motion compensating unit (23) for selecting 
one of forward direction prediction, backward 
direction prediction, bidirection prediction and 

55 background prediction in units of an object 

using reference images read out from frame 
memories of said plurality of frame memory 
groups (43, 44) to generate a motion compen- 
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sated image. 

10. An image coding apparatus as set forth in claim 8, 
wherein re-writing of image contents of a region in 
which an object which is a subject of coding is 5 
included in said plurality of frame memory groups 
(43, 44, 49) in which coded images of the object in 
the past are stored is performed after a certain 
interval of time or in response to a control signal 
from the outside. 10 

11. An image decoding apparatus as set forth in claim 

9, wherein re-writing of image contents of a region 
in which an object which is a subject of decoding is 
included in said plurality of frame memory groups is 
(43, 44, 49) in which coded images of the object in 
the past are stored is performed after a certain 
interval of time or in response to a control signal 
from the outside. 

12. An image coding apparatus as claimed in claim 8 or 

10, wherein searching ranges for a motion vector 
from reference images from said plurality of frame 
memory groups (43, 44, 49) for the individual 
objects are variable for the individual objects. ss 

1 3. An image coding apparatus as claimed in claim 8 or 
10, further comprising differential vector generation 
means (47) for holding a motion vector in the past 
obtained by referring to images from said plurality of 30 
frame memory groups (43, 44, 49) for the individual 
objects and calculating difference vectors sepa- 
rately for the individual objects, wherein the differ- 
ence vectors are variable length coded. 

35 

14. An image decoding apparatus as claimed in claim 9 
or 11, further comprising a motion vector adding 
unit (48) for holding decoded motion vectors in the 
past obtained by referring to images in said plurality 

of frame memory groups (43, 44, 49) for the individ- 40 
ual objects for a certain period of time and adding 
the motion vectors decoded in the past to the 
decoded difference vectors to regenerate motion 
vectors for the individual objects. 
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